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INTRODUCTION 



Development Process for the Assessment Objectives 

In 1969 the National Assessment of Educational Progress (NAEP) was 
established for the purpose of coUecting information about the perfor- 
mance of 9-, 13-, and 17-year-old Americans in a variety of subject 
areas. Over the last two decades, information has been collected about 
performance in mathematics, writing, science, music, art, and social 
studies, as well as other disciplines. Assessments in tnathematics were 
conducted duiing the school terms ending in 1973, 1978, and 1982, 
and a fourth is being held during 1986. 

For each of the assessments, a process was used to collect informa- 
tion from educators and lay citizens to ascertain the content and 
trends in mathematics at that time. This information was then used to 
develop or revise assessment objectives. As educators' and re- 
searchers* views of mathematics have changed over the last decades, 
so has the assessment of mathematics. In addition, as needs have 
emeiiged for information for vaiying purposes — making policy deci- 
sions, understanding student competencies more cleaiiy, determining 
relationships between subject areas — so too have the assessments 
changed. The objectives for each assessment have reflected these 
changes, yet they have preserved the underlying premise: that they 
should reflect the thinking of a wide range of individuals vsdxh an inter- 
est in mathematics. Therefore, parents and other members of the gen- 
eral public, as well as educators, researchers, and teacher educators, 
have participated in the objective-setting process. 

A list of objectives for the first NA£P mathematics assessment was 
developed throu^ detailed planning and study by mathematicians 
and mathematics educators and with reviews by panels of interested 
lay citizens. The resulting statement of objectives was compared to 
other statements of objectives that had appeared in the mathematics 
literature during the preceding 25 years and were found to be consis- 
tent with them. This outcome was both desired and expected since 
one criterion for the National Assessment objectives was that they be 
central to prevailing teaching efforts. A booklet containing the final set 
of objectives was published in 1970. 

Objectives and exercises for the second assessment were devel- 
oped through a series of conferences. Participants included college or 
university matheciatics educators, mathematics classroom teachers, 
and interested lay citizens. An advisoiy board was then formed and 
was instrumental in oi^ganizing the final set of objectives, planning the 
development of exercises, selecting the final exercises, and planning 
subsequent reports. 
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Similar procedures were used to develop the third assessments 
objectives and exercises. The objectives w^ere based on the framework 
used for the second assessment, with some revisions that reflected 
content changes and trends in school mathematics. A nine-member 
advisory committee was instrumental in reviewing the objectives and 
giving guidance throughout the developmental process. 

As has been the case in earlier assessments, the objectives for the 
1985-86 assessment were derived through a process of review and re- 
vision. Studies were conducted to determine the needs, opinions, in- 
terests, and priorities in mathematics education. First, 25 mathematics 
educators and classroom teachers reviewed the objectives used for 
the previous assessment (1981-82). Their responses were collated by 
staff members and were given to the M;:thematics Learning Area Com- 
mittee (See Appendix A) for use in updating the list of mathematics 
objectives for the 1985-86 assessment. This committee was also given 
the NAEP Assessment Policy Committee s request that all 1985-86 as- 
sessment areas (reading, mathematics, science, and computer compe- 
tence) focus on higher-level critical-thinking skiUs. The draft of objec- 
tives prepared by the committee was reviewed and additional 
comments and suggestions were submitted by another panel of 25 
mathematics educators and classroom teachers. These responses, col- 
lated by staff members, were used by the ^f athematics Learning Area 
Commi^ee in preparing the final list of assessment objectives. 
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7 FRAMEWORK FOR THE OBJECTIVES 



The Description of What Is Assessed 

The mathematics objectives are oi^anized into five broad areas: 
(A) Problem Solving/Heasomng 
(6) Routine Application 

(C) Understanding/Comprehension 

(D) SkiU 

(£) Knovdedge 

Since all of the objectives involve elements of problem solving and 
since the Assessment Policy Committee requested an emphasis on 
critical thinking and problem solving, the relationships among the five 
process areas of the objectives can be seen in the following diagram: 




The content is primarily from elementaiy and secondaiy school 
mathematics up to but not including the calculus. The categories of 
mathematics content assessed are: 

(1) Rmdamental Methods of Mathematics 

(2) Discrete Mathematics 

(3) Data Oiganization and Interpretation 

(4) Measurement 

(5) Geometiy 

(6) Relations, Functions, and Algebraic Expressions 

(7) Numbers and Operations 

The Objectives Frameworic for the Fourth Mathematics Assessment 
in Figure 1 shows the relationship between the process areas of learn- 
ing and the mathematics content. The process and content areas are 
described in the following sections of the booklet. 
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Figure 1 

Objectives Framework for the Fourth Assessment 
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Process Areas 

(A) Process Area: Problem Solving/Reasoning 

This categoiy of exercises is intended to assess higher order think- 
ing skills. Therefore, the exercises require processes that are intellec- 
tually more complex than the application of skills or the understand- 
ing of a single concept. 

In the area of problem solving, the exercises require such pro- 
cesses as identifying and using a problem-solving strategy, screening 
relevant from irrelevant information, fonnulating a problem or select- 
ing a model of a problem situation, determining what information 
would be needed to solve a problem* or oi^anizing given information 
to represent the problem. The categoiy also includes such processes 
as formulating generalizations or testing their validity, recognizing pat- 
terns and desoibing or symbolizing tiie relationships, or informally 
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making inferences. In contrast to exercises in the categories of routine 
problem solving and understanding, an exercise in this category might 
ask the student to identify afl needed information to solve a non-rou- 
tine problem. 

(B) Process Area: Routine Application 

Routine mathematical application refers to the use of mathe- 
matical knowledge, sldU, and understanding in solving problems that 
are routine in the sense of familiarity — similar problems would have 
been studied either in the course of instruction or in a textbook assign- 
ment. The student is thus presumed to have had experience in solving 
comparable problems, and transfer to new situations is minimal. That 
is, uiiUe the student is not told how to solve the problem, the stimulus 
is such that selection of an appropriate procedure is almost auto- 
matic. £xercises assessing routine application do not vaiy much from 
textbook problems. An exercise mi^t require, for example, the solu- 
tion of a standard problem on proportion, the demonstration that two 
geometric figures are congruent, or an estimate of the amount of car - 
pet needed for a room. 

(C) Process Area: Understanding/Comprehension 
Mathematical understanding, or comprehension, refers to the in- 
terpretation and elaboration of underiying concepts, assumptions, re- 
lationships, and the like. These underpinnings may be as elementaiy 
as the concept of a fraction or as sophisticated as the concept of a 
deductive system. Understanding does not rely on memory alone, but 
also includes the association of ideas and the perception of 
relationships. 

Exercises assessing understanding/comprehension may require 
the student to identify an example (or something that is not an exam- 
ple) of a concept, to recpgnize when a particular technique may (or 
may not) be helpful, to give an ejqplanation, or to translate from one 
mode of esq^ression to another. For example, a student may be given 
partial information and be asked to identify the additional information 
needed in order to solve a routine problem. 

iD) Process Area: Skill 

Mathematical skill refers to straj^tfbrward, routine manipulation 
and relies on standard procedures that lead directfy to answers. Exer- 
cises assessing mathematical skill asstime tfiat the recpiired procedure 
has been learned and practiced. They do not require the student to 
decide v\Mch procedure to use or to appfy the procedure to a new 
situation. Such exercises aim at measuring proficiency in canying out 
a procedure rather than the understandii^ of how or why it works. 
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Mathematical skill is assessed by exercises that require the perfor<> 
mance of specific tasks such as making measurements, multiplying 
two fractions, performing mental computations, graphing a linear 
equation, or reading a table. 

(E) Process Area: Knowledge 

Mathematical knowledge refers to both the recall and recognition 
of mathematical content as expressed in words, symbols, or figures. 
Mathematical knowledge as described for chis assessment relies> for 
the most part, on inemoiy; it does not ordinarily require any more 
complex mental processes. 

Exercises that assess mathematical kno\A4edge require that a stu* 
dent recall or recognize one or more items of information. Exercises 
involving recall mi^t ask for a multiplication fact> such as the product 
of five and two, or for the statement of a mathematical relation such as 
the law of cosines. An exercise involving recognition might present 
several symbols and ask which symbol means "parallel." 

The distribution of exercises by age group and process area is 
shown in Figure 2, 



FigareZ 

Approximate Percentage Distribution* ofEy.ercises 
by Age and Process 
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*Percents may not total 100 due to rounding. 



Content Areas 

(1) Content Area: Mathematical Methods 

Exercises in this content category assess an understanding of the 
tools of mathematics itselt those processes that are central to the ex- 
tension and development of mathematics and to its use. These meth- 
ods cut across aU the content 2u*eas of mathematics and help to verify 
the various subject matter components. These processes are hi^ily 
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specific to, though not always unique to, the nature of mathematics 
and are essential to mathematical applications in the solution of prob- 
lems, both within mathematics and in other disciplines. They are as- 
sessed as objects of study and goals of instruction. Included are con- 
cepts of deductive and inductive proof logic, models, structure and 
system, routine procedures, problem- solving strategies and empirical 
induction. 

(2) Content Area: Discrete Mathematics 

This content area reflects an increasing awareness in the school 
mathematics curriculum of the widening role of probability, permuta- 
tions and combinations, and linear algebra in modeling applications 
that occur in many different subject areas. These exercises assess pre- 
dictions of outcomes, mathematical expectation, permutations and 
combinations, applications of sums and products of matrices, and 
solutions of matrix equations. 

(31 Content Area: Data Organization and Interpretation 

This content area was made into a separate category in this assess* 
ment because of its emphasis in the school mathematics curriculum 
and its increased usage in other disciplines. The exercises assess 
organizing, analyzing, and interpreting data including determining 
measurement of central tendency and of spread, 

(4) Content Area: Measurement 

The exercises in this content area assess developmental concepts 
of measurement, equivalent measurements, selection and application 
of appropriate size and type of units, instrument reading, and preci- 
sion and accuracy. Measurements and applications of measurements 
of length, time, tiamperature, massAveight, area, volume, capacity, and 
angles are included as well as measurement applications to circles, 
scale drawings, and money. 

(5) Content Area: Geometry 

Exercises in this category assess properties and relations of 
geometric figures such as parallel lines, peipendicular lines, similar 
polygons, congruent figures, vertical angles, angles of a triangle, and 
measures of central an^es of a circle as well as the Pythagorean rela- 
tion. Relations established by formal proofis such as various loci of 
points, an^es inscribed in semicircles, and special li^t triangles are 
included. Also assessed are lines of symmet^ and images of figures 
imder flips, turns, and slides, and other spatial relations in two and 
three dimensions such as intersections of planes and solids. 
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(6) Content Area: Relations, FunctionSf and Algebraic Expressions 
This content area is broad in scope. Exercises assess the use of 

variables in expressions of relationships; translations from words to 
symbols; and use of variables to represent properties of operations, 
number theory concepts, and properties of equality and inequality. 
Determining solutions of equations and inequalities including systems 
of equations and quadratic equations is assessed. Generalization of 
patterns; evaluation and interpretation of functions and formulas; 
plotting of graphs in rectangular and other coordinate systems, and 
exponential and trigonometric functions are also included in this 
content area. 

(7) Content Area: Numbers and Operations 

In this content area concepts of numeration and number are 
assessed for w^hole numbers, common fraction ^s, deciiV^al fractions, 
integers, and percents. Operations with these numbers, including 
mental computation, are assessed as is estimation of computation. 
Number properties and relationships such as number patterns and 
ratio and proportion are assessed in this content area. 

An outline of these content areas is provided in Appendix B. The 
distribution of exercises by content category and age group are shown 
in Figure 3, 



Figured 

Approximate Percentage Distribution* of Exercises 
by Age and Content 
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Problem Solving 

Problem solving/reasoning is one of the categories iii the process areas 
and has been addressed; how^even problem solving, in its broadest 
meaning, was incorporated into all content areas of this mathematics 
assessment. More than just the abili^ to select solutions of routine 
word problems is to be assessed. The Mathematical Learning Area 
Committee prepared a brief description of skills and abilities needed 
in problem sohing,The description is provided here as further clarifi- 
cation. It is not intended to be all-inclusive or a set of objectives. 

Brief Description of Problem-Solving 
Skills and Abilities 

Pre-analysis skills such as: 

• the identification of mathematical problems or mathematical 
questions that could arise from a given descriptic.i of a practical 
situation or a given mathematical model (graph, diagram, table, 
system of equations, etc) 

• the formulation of reasonable mathematical hypotheses from 
given information or a general description of a situation 

Analysis skills such as: 

• the identification of knovm facts, unknowns, or questions in a 
given problem situation 

• the identification of information/data needed to solve a given 
problem, any extraneous information, and the interpretation of 
technical terms in the problem 

• the recognition of problems in which the underlying mathe- 
matical processes are the same as a given problem, but the con- 
text is different 

Ability to select viable strategies for the solution of a given problem 
such as: 

• the identification of general strategies (appropriate drawings, 
graphs, tables, simpler problems, patterns, "guess and check," 
"woricing backwards," etc J that may help in the solution of a 
problem 

• the identification of specific procedures (sequence? of steps and/ 
or operations) that wOl lead to a solution of a problem 
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• the use of estimation to predict a reasonable solution for a given 
problem 

Ability to interpret the solution of a problem such as; 

• the recognition and/or verification of a sensible solution for a 
given problem 

• the identification of new relationships, or the prediction of possi- 
ble consequences based upon the solution of a given problem or 
problems 

Attitudes 

NAEP has assessed attitudes toward mathematics beginning with the 
second assessment in 1977-78. Five cathodes of attitudinal measures 
were developed and have been used: (1) mathematics in school (2) 
mathematics and oneselC (3) mathematics and society (4) mathematics 
as a discipline, and (5) attitudes toward computers. 

Cakndator 

Because of the increasing availability and populari^ of calculators, 
NAEP has gathered information about their use by students beginning 
with the 1977-78 mathematics assessment. 

The assessment includes five categories of exercises. They are: (1) 
routine computation, (2) more difficult computations, (3) understand- 
ing computers, (4) eaqiloration, and (5) application or problem solving. 
Some calculator activities, such as understanding and exploration, are 
more appropriate for instructional use in the classroom and are not 
emphasized in the assessment. Thus, of the five categories, computa- 
tion, nonroutine computation, and appUcation are measured in the 
greatest depth. Additional information on these categories can be 
found in ^pendix C. 

A minimal amount of instruction on the use of the calculator is 
given prior to the exercises. Many of the exercises are repeated, but 
without the use of calculators, in other parts of the assessment for the 
same age group to permit comparisons of performance with and with- 
out the calculator. 

Background information is gathered from each student as to expe- 
rience with calculators. The questions include how often the student 
has used a calculator, if the student's fiunily owns one, in wiiat courses 
the student has used a calculator, and what experiences with cal- 
culators the student has had outside of school. 
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Questions to be Addressed 
fay the Assessment 

In developing and selecting exercises for the assessment, care was 
taken to ensure an appropriate balance of emphasis on both the con- 
tent and process dimensions. NAEP previously oi^ganized questions 
according to the process and attitude dimensions and included the 
topics of the content dimension. The questions were based on pri- 
orities of the mathematicians, mathematics educators (including 
teachers), educational administrators, and lay people involved in the 
developmental process of the assessment. The following questions in- 
corporate the revisions made by the Mathematics Learning Area Com- 
mittee for the 1985-86 assessment and provide an overview of the 
assessment. 

A. Mathematical problem solving/reasoning 

1. How well can students analyze a situation and formulate a 
mathematical problem? 

2. How well can students screen relevant from irrelevant infor- 
mation in a problem situation? 

3. How weU can students oi^ganize given information in a prob- 
lem situation into a usable form (table, graph, chart, diagram, 
or algebraic esqjressions, etc.)? 

4. How well can students identify strategies and procedures that 
may help in solving a problem? 

5. How well can students carry out procedures selected for solv- 
ing a given problem? 

6. How well can the students check for reasonableness of an an- 
swer that resulted from a procedure used to solve a problem? 

B. Routine mathematical application 

1. How well can students solve a routine textbook problem? 

2. How well can students select a procedure to solve a routine 
application? 

3. How well can students apply problem-solving strategies? 

4. How well can students estimate an answer to an application 
problem? 

5. How well-can students interpret data and draw conclusions? 

6. How well can students use mathematics, including logic, in 
reasoning and making judgments? 
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7. How well can students use a calculator to solve an application 
problem? 

C. Mathematical understanding/comprehension 

1. How well can students translate a verbal statement into sym- 
bols or a figure, and vice versa? 

2. How well do students understand mathematical concepts and 
principles? 

3. How well can students see relations between the information 
and questions in a problem situation and simultaneously be- 
tween those two elements and the concepts, principles, and 
processes previously acquired? 

4. How well can students select and draw on resources such as 
paper and pencil, computer; calculator; or estimation? 

D. Mathematical skiU 

1. How well can students perform paper-and-pencil computa- 
tions, including computations with wiiole numbers, integers, 
fractions, decimals, percents, and ratios and proportions? 

2. How well can students perfoim algebraic manipulations? 

3. How well can students perform geometric manipulations such 
as constructions and spatial visualizations? 

4. How vi^ll can students make measurements? 

5. How well can students read graphs and tables? 

6. How well can students compute statistics, probabilities, per- 
mutations or combinations? 

7. How well can students jjerform mental computations, includ- 
ing computation vAth whole numbers, lections, decimals, and 
percents? 

8. How well can students estimate measurements and answers to 
computations? 

9. How well can students perform co-^putations involving whole 
numbers, decimals, fractions, anr ^^rcents using calculators? 

10. How well can students read flow charts or simple computer 
programs? 




E. Mathematical knowledge 

1. How well can students recall and recognize facts, definitions, 
and symbols? 

E Attitudes 

1. How do students feel about the mathematics tt ey encounter in 
school? 

2. How do students feel about the various activities in mathemat- 
ics classes? 

3. How do students fee? about their personal e;cperience with 
mathematics? 

4. What are students' beliefs about the nature of mathematics as a 
discipline? 

5. What are students' beliefs about the value of ma'Nematics to 
society? 

6. What are students' beliefs about computers? 
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APPENDIX B 



Content Oudine 

The following list is an outline of content for which the test questions 
were written. Not all the subtopics are tested at eveiy age level. Stu- 
dents are not tested on the meanings of these terms. Questions are 
stated in the students' language and at the students' level of maturity. 

1. FUndaunental Methods of Mathematics 

a. Models 

b. Problem-solving strategies (heuristics) 

c. Informal induction 

d. Informal deduction 

e. Routine procedures (algorithms) 

f. Axiomatic systems 

g. Logic 

2. Discrete Mathematics 

a. Probability (simple, compound, and independent events; 
odds; mathematical expectation; 

b. Permutations and combinations 

c. Matrices (sums and products of matrices, matrix equations) 

3. Data Oi^anization and Interpretation 

a. Oi^anizing, displf^dng, and interpreting information (tallies, 
tables, charts, and graphs) 

b. Measures of central tendency (mean, median, mode) 

c. Measures of spread and position (range) 

d. Sampling and polling (effects of sampling, random samples, 
opinion polls) 

4. Measurement 

a. Concepts of measurement (nonstandard units, iteration of a 
unit model, additive property) 

b. Standard units of measure (appropriate size and type of 
unit, conversions within a system, estimation of measure- 
ments) 

c. Measuring instruments (reading of instruments, precision/ 
accuracy of measurement) 

d. Applications of measurement (time, money, perimeter, cir- 
cle, scale drawings, etc.) 
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5, Geometiy 

a. Geometric figures and their properties (polygons, solidSi cir- 
cles, angles, curves, lines, line segments, rays) 

b. Relations established by definition (parsdlelism, perpen- 
dicularity similarity congruence) 

c. Relationships established by theorem (informally presented: 
angles opposite congruent sides of triangle, vertical angles, 
sum of aJigles of triangle, Pythagorean relation, measures of 
central angles of a circle; formal proof: loci, parallel lines, 
parallelograms, angle sum for n-gon, inscribed quad- 
rilaterals, angles inscribed in a semi-circle, special right tri- 
angles, constructions) 

d. Motion geometry (informal: lines of S3aTimetry; flips, turns, 
and slides) 

e. Coordinate systems applied to geometiy 

f Spatial visualization (orientation in S-dimensional space, de- 
composition of irregular shapes into fanr^'liar shapes, inter- 
sections of point sets — in 2-dimensional und 3-dimensional 
spaces) 

6. Relations, Functions, and Algebraic Expressions 

a. Use of variables (translatioit from veii)al to sya.bolic; repre- 
sentations of properties of operations, equalities, and in- 
equalities; equivalent equations and inequalities; simplifica- 
tion of algebraic expressions) 

b. Simple relations and functions (generalization of patterns, 
direct and indirect variation, evaluation of formulas and 
functions, domain and range of functions) 

c. Solving equations and inequalities (simple equations, in- 
equalities, systems of equations, quadratic equations) 

d. Coordinate systems (rectangular, polan and 3-dimensional) 

e. Flinctions (pre-calculus: properties of functions and inverse 
of a function; graphs of functions and relations: algebraic, 
exponential, and logarithmic) 

t Trigonometry (identities, radian measure, solution of tri- 
angles, equations, graphs of functions) 




Numbers and Operations 

a. Numeration (rounding, wiiole numbers, fractions, decimals, 
percents, integers, scientific notation) 

b. Number concepts (order, equivalence, number line, vviiole 
number, fraction, decimal, percent, integer, changing a rep- 
resentation of a number from one form to another) 

c. Operations (fundamental operations: concepts, wiiole num- 
bers, fractions, decimals including percents, integers, com- 
binations; inverse operations; povvers and roots) 

d. Mental computation 

e. Estimation (computational and applications) 

f. Properties 

g. Relationships (number patterns, ratio and proportion, and 
inequalities) 
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APPEIVDKC 



Caknilator Usage 

The use of the calculator has been included in the National Assess- 
ment of Educational Progress beginning with the second assessment. 
At that time the availability and popularity of calculators made it 
important for the National Assessment to collect iriformation on their 
use by students. With the widespread use of ine?^nsive calculators, 
the mathematics curriculum oontiDuefi to be imfiuenced by their use. 

Several conferences were oiiganized by NAEP to discuss tlie place 
of calculators in the seccnd assessment. Five categories of exercises for 
which calculators nU^t be used in the classroom or for assessment 
were identified. These categories are: 

1. Routine Computations — Computations with whole numbers, 
decimals, Actions, and integers that are typical of the mathe- 
matics curriculum could be included, not to replace computa- 
tion as taught in the past, but to include the use of calculators as 
tools of mathematics. 

2. More D^cult Computations — Students could be asked to per- 
form more difficult computations than the routine, or computa- 
tions for which algorithms are not formally taught. For example, 
nine-3^ear-olds mi^t be asked to do computations with deci- 
mals. Thirteen-year-olds might be given questions requiring 
conversion between fractions and decimals. Students at all ages 
could be given exercises requiring work v^th laige numbers or 
complicated decimals that would make computation v^thout a 
calculator tedious. 

3. Understanding Concepts — Nine-year-olds might use the cal- 
culator to learn more about place value, and Id-year-olds might 
learn estimation for order of magnitude. Seventeen-year<>olds 
mi^t use the calculator to facilitate learning order of opera- 
tions, extracting square roots, graphing functions, and under- 
standing properties of functions. 

4. Epcploration- -Number theory could be included. Problems in- 
volving serie s, summations, patterns, or divisibility are typical in 
this category. 

5. Applications and Probl&n Solving — Routine and more difficult 
word problems, such as multistep problems, would be included 
in this cat^ory. For nine-yearolds, the use of the calculator per- 
mits some problems that require larger numbers than those to 
which students are accustomed. Problems for 13-year-old8 




might involve percent, unit pricmg, and more difficult word 
problems. Sevenleen-year-oidik might be given a \'ariety of real- 
istic consumer problems as weU as exercises requiring the use 
of mathematical formulas. 

The five cat^ories could be assessed at each of the thi^ age levels; 
hovwever, practical considerations impose limitations on what can be 
assessed. Exercises in some categories, such as understanding con- 
cepts and exploration, are more appropriate for instructional pur- 
p)oses than for assessment. Therefore, categories 1, 2, and 5 are mea- 
sured in the greatest depth. 

Some exercises are given uath and without calculators to provide 
data on student performance with and without the aid of the 
calculator 




